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ABSTRACT 



Demographic and institutional factors responsible for the 
large variations of student retention and achievement at further education 
(FE) colleges throughout the United Kingdom were examined. Data from 
individualized student records were analyzed, and 10 FE colleges identified 
as serving areas of deprivation were visited to assess the extent to which 
demographic factors are the main determinants of differences in rates of 
retention and achievement and to identify institutional practices connected 
with higher and lower retention and achievement. Although demographic 
differences in student profiles had an impact on the relative rates of 
achievement recorded by different colleges, demographic make-up of student 
bodies did not appear to explain the major part of interinstitutional 
variations in achievement . A substantial part of the achievement gap appeared 
to stem from factors lying within the direct influence of FE colleges. A 
total of 53 key characteristics associated with effective interventions to 
raise student achievement were identified. The characteristics involved the 
following aspects of practice: strategic commitment; collegewide approaches; 
recruitment, placement, and induction; design and delivery of curriculum; 
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tutorial and other support; teaching and pedagogy; and monitoring, 
evaluation, and follow-up. It was concluded that more work is required to 
determine how demographic factors act as barriers to achievement. (Contains 
12 references.) (MN) r 
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PREFACE AND ACKNOWLEDGEMENTS 



The research project that this report concerns was undertaken as part of the Raising 
Quality and Achievement programme, and sponsored by the Department for 
Education and Employment (DfEE). Work on the project commenced in February 
2000 . 

The report aims to clarify the relative influence on student achievement of 
institutional performance and of demographic factors, and to guide colleges on 
effective strategies to raise achievement. Its contents are devoted primarily to the 
findings arising from the second stage of the project, during which the Agency’s 
consultants visited a sample of colleges with a range of achievement rates. It also 
contains a brief summary of the outcomes of the first stage of the project, which 
investigated the extent to which inter-institutional differences in FE-sector 
achievement rates can be explained by relative differences in the profiles of enrolled 
students, as recorded in the Further Education Funding Council’s (FEFC’s) 
Individualised Student Record (ISR). For full details of the Stage 1 findings, readers 
are referred to Differential achievement: what does the ISR profile tell us? (Davies 
and Rudden 2000). 

In undertaking this work, the Learning and Skills Development Agency 
acknowledges the considerable assistance received from the managements and 
staff of the participating colleges, the FEFC, and the Responsive College Unit 
(RCU), without which this report would not have been possible. 



Project team 

Project Leader Peter Davies 

Consultants Tricia Bailey 

Diana Brooke 
John Maynard 
Geoffrey Petty 
Stephen Sawbridge 



Statisticians 



Terry Rudden 
Simon Williams 



SUMMARY OF FINDINGS 

Demographic influences on achievement 

• Demographic differences in student profiles have an impact on the relative rates 
of student achievement recorded by different colleges. Generally speaking, there 
is a negative correlation between measures of deprivation and overall rates of 
achievement. 

• In itself the demographic make-up of student bodies does not appear to be 
capable of explaining the major part of inter-institutional variations in 
achievement. Colleges with the lowest rates of student achievement are below 
average primarily because their rates are lower for a// types of student, not 
because they have higher than average proportions in categories associated with 
below average achievement - even though the latter is usually also the case. 

• Once variations in the Individualised Student Record (ISR) profiles have been 
accounted for, there is no evidence that the remaining differences in college 
achievement rates are connected in any marked degree to variations in students’ 
prior attainments, or in the numbers of qualifications for which they are entered. 

• Initial analysis of student bodies by postcode reveals no evidence that 
deprivation has a significant impact on achievement over and above that 
revealed by ISR comparisons. For every additional 10% of total student numbers 
from deprived areas, overall achievement rates fall by around 3% (1997-98 data) 
or less (1998-99). This relationship between achievement rates and the 
percentages of students from deprived areas is relatively weak. Even if the 
correlation were strong, it would not account for the bulk of the inter-institutional 
achievement gap. 

• Demographic variations appear to account at most for no more than half of the 
inter-institutional differences in student achievement, according to the measures 
currently available. 7\// colleges with overall achievement rates below the average 
should be capable of improving them to the current median level, at the very 
least. Further investigations of the influences of deprivation and ethnicity are 
needed before we can fully exclude the possibility that they could explain a 
significant further element of the remaining achievement gap. 



Institutional performance 



• A substantial part of the achievement gap appears to stem from factors that lie 
within the direct influence of colleges - that is, as a result of differences in 
institutional ethos, systems, procedures and practices that affect student 
performance. Evidence already assembled concerning influences on student 
retention, together with the reports from the FEFC Inspectorate, and from the 
National Audit Office, supports this hypothesis. 



• Colleges with low achievement rates are not confined to those serving areas of 
high deprivation. Year-on-year improvements have generally been most 
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pronounced among the colleges with the lower achievement rates - a pattern 
consistent with the hypothesis that a significant element of student achievement 
is directly susceptible to college-level interventions. 

• Our study of 1 0 colleges that formed Stage 2 of this project suggests that, in 
general and taking account of the variations in the profiles of the student bodies, 
those with the better performance in terms of overall achievement rates during 
the period 1996-99 had for whatever reason made the improvement of retention 
and achievement a major strategic priority at an earlier stage and had been more 
successful at embedding a culture of continual improvement at all levels. In these 
cases there was evidence extending over a longer period of: 

> the high priority accorded to raising student achievement within the 
college strategic plan 

> consideration of the issue by the SMT and governing body 

> the establishment of achievement targets, against which progress was 
monitored, and the use of benchmarks 

> effective communication to staff of the central importance of raising 
student achievement 

>. a culture of high expectations of both staff and students 

> the allocation of responsibility and authority at curriculum leader and 
course team level 

> regular monitoring and action-planning via face-to-face meetings 
between managers and staff 

> encouragement of bottom-up initiatives within a college-wide strategy 
framework 

> review of teacher performance, supported by tailored staff 
development. 

• A number of the most interesting innovations are now in place in colleges with 
the greatest cause for concern about their achievement rates. 



Raising student achievement 

• Key characteristics associated with effective interventions to raise student 
achievement were identified as: 

Strategic commitment 

> a clear commitment at SMT and governing body level to the importance 
of delivering and sustaining high levels of student achievement 

> the establishment of clear, challenging but realistic achievement 
targets, against which progress can be monitored 

> effective communication to college staff of the importance accorded to 
student achievement, and to the associated targets 

> a culture of continual improvement, underpinned by clear 
accountabilities. 

> an embedded belief that the college could make a significant difference 
to student achievement, even allowing for the influences of 
demographic factors outside its direct control. 
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College-wide approaches 

> annual course review and self-assessment procedures, with the active 
involvement of curriculum leaders and course teams, designed to 
identify strengths to be built upon and weaknesses to be addressed 

> action plans arising from the reviews, containing strategies to tackle 
weaknesses and meet targets 

> regular monitoring of retention and achievement in relation to action 
plans, and clear accountability and reporting on follow-up 

> a feed-through from course review into teaching and tutorial practice, 
including associated programmes of staff development 

> an expectation and encouragement of bottom-up, as well as top-down 
initiatives to improve the design and delivery of the curriculum 

> securing parental involvement 

> recognition that there is no one foolproof way to raise achievement, but 
that a range of strategies has to be followed, simultaneously and 
continually. 

Recruitment, placement and induction 

> effective communication of the curriculum offer by a range of media, 
with an emphasis on clearly setting out course requirements and 
progression opportunities 

> well-developed advice and guidance services for prospective students. 

> clear entry criteria, applied systematically 

> enrolment onto generic learning pathways, with final choice of course 
made later in the first term 

> regular review of course placement in relation to the curriculum offer 
and pre-enrolment information, to ensure appropriate choice of clear 
learning pathways from foundation level upwards 

> early diagnosis of 'at risk’ students, to identify and alert the need for 
learning and other support 

> the availability of ‘taster’ sessions and the ability of students to make 
early-stage course transfers 

> well-planned induction programmes, with clearly set out and high 
expectations of what would be required of students in terms of their 
commitment 

> careful selection of staff involved in the process, supported by tailored 
programmes of training. 

Design and delivery of curriculum 

> continual adaptation of curriculum portfolio in light of changes in the 
profile of the student body, and movements in the labour market 

> a willingness to withdraw from areas of provision with achievement 
rates persistently below target 

> the availability of a full 'curriculum ladder’ allowing progression right the 
way through from pre-entry to Level 4 and above 

> a widespread practical awareness of the concept of inclusive learning 
manifested in differentiated approaches to the needs of individual 
learners 

> a pro-active approach to ensuring the take-up of additional learning 
support by those who required it 
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> high expectations of students, based on teaching staff who provided 
good professional role models 

> on vocational courses, the integration of related work experience and 
effective workplace simulation, or both 

> coordinated scheduling of coursework to eliminate bottlenecks 

> review and adjustment of the timetable in relation to students’ other 
commitments. 

Tutorial and other support 

> regular review of students’ academic progress and personal 
circumstances, individually as well as in groups 

> the negotiation with individual students of targets and action plans to 
achieve them, making use of value added systems wherever possible 

> close links between tutors and teaching staff 

> pro-active referral to flexible systems of additional learner support, 
wherever necessary 

> close liaison between tutors and Student Services, and other sources 
of support on non-academic matters 

> rigorous systems for the selection, training and appraisal of staff with 
tutorial responsibilities. 

Teaching and pedagogy 

> active involvement of teaching staff in course review procedures 

> teachers accept appropriate accountability for student achievement 

> rigorous, regular and consistent system of lesson observation 

> utilisation of observation reports within agreed systems of staff 
appraisal 

> culture of ongoing improvement, supported by individually tailored 
programmes of continual professional development 

> identification and dissemination of effective practice. 

Monitoring, evaluation and follow-up 

> MIS capable of providing accurate and up-to-date information on 
retention and achievement, down to course level and by individual 
student, and in a user-friendly format 

> readily available online access to MIS on the part of managers and 
teaching staff 

> all staff trained in accessing, interrogating and making use of MIS data. 

> clearly stated policies on attendance and punctuality, with immediate 
follow-up of unexplained absence 

> culture of student obligations as well as entitlements 

> early identification of problems caused by outside pressures, and offer 
of support 

> direction and programming of self-study time, especially in the case of 
younger students 

> involvement of parents or other family members in sustaining student 
support. 
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The way forward 

• Some areas remain less developed than others: 

> MIS, where despite improvements there is still a wide gap between the 
best practice and the level of development that is typical 

> changes in course delivery in response to the more rigorous 
identification of retention and achievement problem areas; concepts of 
inclusive learning still have some way to go before they are likely to 
impact fully on the majority of students 

> sharing and learning from effective practice: colleges seem much better 
attuned to identifying weaknesses and acting upon them, than to 
building on their own strong points. 

More work is required to understand fully the ways in which demographic factors act 
as barriers before we can be certain that their impact is not greater than we believe 
on the basis of the evidence currently available. At present it appears that the efforts 
at college level are already bringing about a noticeable narrowing in the inter- 
institutional achievement gap. Such efforts place considerable demands upon 
colleges. Ultimately, though, we believe that it ought to be possible to reduce the 
achievement gap to half its present size, or less. 
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1 BACKGROUND AND AIMS OF PROJECT 



In the drive to raise standards and improve student achievement across the FE 
sector, it has been recognised for some time that there appeared to be wide 
differences in the retention and achievement rates of different colleges that were not 
obviously explicable by relative differences in student demography, or in other 
factors outside the direct influence of colleges (Audit Commission and Ofsted 1993). 
FEFC has undertaken analysis which indicates that retention and achievement rates 
for 41 general FE and tertiary colleges serving areas of deprivation (so-called ‘high 
Widening Participation (WP) Factor’ colleges) were found to be significantly below 
those of other colleges of the same type. No other group showed such significant 
differences as these. However, even within the high WP Factor group there is a 
considerable range of achievement rates (FEFC 1999). 

In recent years the Agency has established robust evidence to demonstrate that 
student retention is influenced substantially by factors within the sphere of colleges’ 
influence. A number of college-level intervention strategies have been shown to be 
capable of effecting significant improvements in retention rates (Martinez 1997; 
Martinez and Munday 1998). A body of case study evidence has been assembled 
which indicates that the same also applies to achievement rates (Martinez 2000). 
Investigations undertaken last year by the National Audit Office (NAO) broadly 
support these findings (Comptroller and Auditor General 2001). 

Nonetheless, the impact of demography and of institutional performance on retention 
and achievement rates was not known with any precision. Without greater certainty 
on their relative influences, colleges remain concerned not to set benchmark targets 
that are unrealistically high. Nor do they wish to devote additional resources to 
tackling achievement rates that are below the average for apparently equivalent 
institutions if, in fact, they are really performing well, once proper allowance is made 
for comparative student profiles. Equally, some colleges with achievement rates that 
are above average for the sector may be lulled into complacency if they do not make 
proper allowance for the particular characteristics of their student profile. 

In this context, therefore, the Agency established a research project Differential 
achievement, with the following aims: 

• to investigate the reasons for the large variations in student retention and 
achievement, within and between institutions, and between different groups of 
students 

• to assess the extent to which demographic factors are the main determinants of 
differences in rates of retention and achievement 

• to identify institutional practices connected with higher and lower levels of 
retention and achievement so as to inform college-level improvement strategies. 

Stage 1 of the project was primarily directed at the first two of the above aims, and 
involved analysis of ISR data. A summary of Stage 1 findings follows in section 2 of 
this report. The full report on the first stage of the project is now published as 
Differential achievement: what does the ISR profile tell us? (Davies and Rudden 
2000 ). 

In the second stage of the project, the Agency’s consultants undertook a programme 
of visits to a sample of high WP Factor colleges to examine systems and practices 
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that might affect achievement. The visits were aimed at identifying the strategies that 
appear most effective in securing good achievement rates. 



2 STAGE 1 FINDINGS 

Impact of ISR profile on achievement rates 

Taken together, the relative differences in student profiles as recorded by the ISR 
are not capable of explaining the major part of inter-institutional variations in 
achievement. Colleges with the lowest rates of student achievement are below 
average primarily because their rates are lower for a// categories of students, not 
because they have higher than average proportions of students in the ISR categories 
associated with below average achievement - even though the latter is usually also 
the case. 

The analysis undertaken in Stage 1 was conducted on the basis of the data for the 
year 1 997-98, the last fully audited set of figures available at the time the work 
commenced. In that year, the six high WP Factor colleges with the lowest long- 
course achievement rates averaged 44%, while the six with the highest rates 
averaged 78%. From our analysis of comparative ISR data we estimate that, at the 
absolute maximum, some 16 percentage points of the 34 percentage point gap 
between the colleges with the lowest and highest achievement rates could be 
attributed to relative differences in student profiles - that is, just under half. In the 
meantime, the 1998-99 data has been released by FEFC. This reveals that the 
achievement gap has closed to 29 percentage points (54-82%), the colleges with 
the lowest achievement rates having raised them at a faster than average rate 
compared with the previous year. Comparisons of the relative ISR profiles of the 
lowest and highest achieving of colleges, on the basis of this latest data, reveal 
notably fewer differences than for the previous year that could account for the gap 
between the two groups. 



Reliability of data 

There are a number of potential explanations for inter-institutional variations in 
achievement in addition to those related to relative differences in ISR profiles. One 
possibility is that the data within the ISR is inaccurate to an extent that exaggerates 
those differences we have identified. So far as FEFC is concerned, in three of the 41 
cases the 1997-98 long-course achievement rates for high WP colleges were 
flagged as being inaccurate, each of which was excluded from our analysis for this 
reason. There are some variations between the 1996-97, 1997-98 and 1998-99 
long-course achievement rates for the same colleges that raise suspicions about the 
reliability of at least one of these year’s figures. For example, one of the colleges that 
lay within the highest achieving group in 1996-97 only just escaped the lowest 
achieving group the following year. Two other colleges journeyed almost as far in the 
opposite direction. Three of the colleges in the highest achieving group in 1998-99 
were well outside in the preceding two years. One of the colleges included in our 
sample for Stage 2 of this project lay within the highest achieving group in 1996-97 
and 1998-99, but within the lowest achieving group in the intervening year! 
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Nonetheless, the general reliability of the ISR is acknowledged to have improved 
over time, and we can detect no evidence that the incidence of any inaccuracies was 
distributed in such a way as to have produced the achievement gap that we have 
noted. 



Qualifications per student 

A further possibility is that the lowest achieving group of colleges tends to enter 
students for a greater than average number of qualifications, while the higher 
achieving group does the opposite. However, when the average numbers of 
qualifications per student are plotted against the achievement rates for all high WP 
Factor colleges, there is little or no correlation. This hypothesis can therefore be 
rejected. 



Other demographic factors 

There are factors that might affect the relative profiles of colleges’ student cohorts in 
addition to those we have used as a basis for our analysis. There are limits to the 
ability of the ISR to measure deprivation as it affects individual students. Other 
measures are available including postcodes, which are widely used as proxy 
indicators of relative deprivation. In as much as this method of measurement is 
successful, it may be said to capture a range of factors related to deprivation. If 
differences in the relative deprivation of their student bodies provided an explanation 
for inter-institutional differences in achievement rates across the high WP group of 
colleges, we should expect there to be a correlation. In fact, the relationship between 
achievement rates and the percentages of students from postcodes that trigger the 
WP uplift is weak. The line of best fit suggests that for every additional 10% of total 
student numbers that trigger the uplift, overall achievement rates fall by around 3% 
(1997-98 data) or less (1998-99). But the weakness of the correlation is revealed by 
the significant minority of high achieving colleges that have student profiles 
comprised overwhelmingly of students who trigger the uplift. 

It is also possible that some of the fields within the ISR provide a level of 
discrimination between different types of students that is insufficient for the purposes 
of assessing the true impact on rates of achievement. For example, the ethnicity 
category Black-African encompasses individuals who are refugees in severely 
deprived circumstances, for whom English is not the first language, along with 
English speakers from largely middle-class backgrounds. In these circumstances, 
there is clearly the potential for the achievement rates of different colleges to be 
disproportionately affected. 



Prior attainment 

This investigation has not examined in detail the relationship between achievement 
rates and prior attainment in nationally recognised qualifications. This is a complex 
field, which the Agency has investigated in another project concerning value added 
[{insert reference)]. It is now generally accepted that there are significant correlations 
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between GCSE scores and performance in A and AS levels. There also appear to be 
correlations between GCSE scores and GNVQ performance, albeit less strong. No 
reliable evidence yet exists as to the relationship between GCSE scores and other 
qualifications. 

The relationship between the overall achievement rates of colleges and relative 
differences in the prior attainment profile is not entirely clear. There is obviously a 
negative correlation between overall achievement and the relative proportions of 
students at each qualification level. Colleges with lower achievement rates tend to 
have relatively higher proportions of students enrolled at Levels 1 and 2, though not 
to an extent that could account for the bulk of the achievement gap. Moreover, as the 
Agency has pointed out elsewhere, in theory the prior attainment profiles of students 
should not impinge further on FE, provided that students are enrolled at the level 
appropriate to their abilities (Martinez 1999). The fact that colleges with the lowest 
rates of student achievement have relatively higher proportions of students at Levels 
1 and 2 does not suggest that they have a greater than average tendency to enter 
students for qualifications at levels beyond which they are able to cope. 



Institutional performance 

Overall, therefore, the investigations undertaken as the first stage of the project 
concluded that the major part of institutional differences in achievement rates could 
not be explained by relative differences in student profiles as recorded in the ISR, or 
in other available measures. The main explanation, therefore, must lie in the 
influence of: 

• geodemographic and other factors outside the direct influence of the college, 
which are not captured by the ISR, or by other measures that are utilised to 
profile student bodies 

• differences in institutional ethos, systems, procedures and practices that affect 
student performance. 

Evidence already assembled concerning influences on student retention, together 
with the reports from the FEFC Inspectorate, and from the National Audit Office, lead 
us to believe that, in general, the latter are likely to account for a substantial 
proportion of the inter-institutional differences in achievement rates, probably at least 
half. 



3 STAGE 2 METHODOLOGY 

Sample of colleges 

Ten colleges cooperated in Stage 2 of the project, selected so as to include a range 
of long-course achievement rates. Five consultants undertook the fieldwork, each 
working with a pair of colleges with different long-course achievement rates, based 
on the 1997-98 data. Details of the colleges concerned are listed in Table 1, 
grouped in the pairs visited by the consultants. Each college was visited twice during 
the period July-November 2000. Members of the senior management teams, 
curriculum managers, teaching staff and students were interviewed. Interviews were 
semi-structured, using a standard pro-forma to ensure consistency. 
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Table 1: Details of colleges participating in Stage 2 oi Differential achievement 
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number of other FE sector colleges within the 15k radius - the larger the variation from the average for this group of colleges, the 
greater the adjustment to the first figure - upwards in cases where the number of other colleges was above average, downwards 
where it ivas below average. 

This figure was flagged by FEFC as a significant underestimate. 



Relevant documentation was also examined before and after each visit, including 
strategic plans, retention and achievement policies and Inspection reports. Detailed 
reports on each college, indicating practices and systems that appeared to influence 
student achievement, were discussed and synthesised. 



Profile of colleges participating in Stage 2 of the project 

Eight general FE/tertiary colleges and two sixth form colleges participated in Stage 2 
of the project.^ In 1998-99, the long-course achievement rates for the 10 colleges 
concerned ranged from 43% to 83%. Over the period since 1996-97, improvements 
in achievement rates were generally most pronounced among the colleges whose 
achievement rates had been lowest at that time. In the case of four of the 
participating colleges, long-course achievement rates declined over the period in 
question, albeit marginally so and from a high baseline in three of these instances. 
The 1997-98 achievement rate for College D appears anomalously low compared 
with the rates for the years on either side, raising doubts about the accuracy of that 
figure. It should also be noted that there are considerable variations in the 
achievement rates of different courses within each individual college. There are 
therefore a number of courses with high rates of achievement at colleges with low 
overall achievement rates, and vice versa. 

Within the sample, the colleges with the lower achievement rates are generally those 
with the higher Widening Participation factors, and the larger proportions of students 
who qualify for the WP funding uplift. However, in line with the findings arising from 
Stage 1 of the project, this link between measures of deprivation and rates of 
achievement is by no means uniform, and does not appear to explain the bulk of the 
‘achievement gap’. 

In order to check if there was any indication that relative success in widening 
participation had a negative effect on rates of achievement, analysis was undertaken 
to examine relative catchment area penetration. Making like-with-like comparisons of 
this kind between different colleges presents considerable difficulties. There is no 
universally accepted definition of college catchment area, and the geographical 
spread of recruitment can differ markedly from one to another. Table 1 displays two 
measures of penetration for each of the 10 colleges. The first is calculated so as to 
avoid underestimation of the ‘true’ penetration rates of the colleges with the widest 
geographical spread of recruitment. The second makes a further adjustment to allow 
for different competitive circumstances, by weighting the first figure upwards or 
downwards according to the numbers of other FE-sector institutions located nearby. 
Nonetheless, even with these modifications, reservations remain about the validity of 
such measures as indicators of widening participation. Given these caveats, though, 
within the sample of Stage 2 colleges there is no obvious sign of any general 
negative correlation between catchment area penetration and achievement rates. 



Eight of the colleges have a high WP Factor. College F is categorised by FEFC as only ‘average’ in 
respect of the relative deprivation of its students. College B is categorised as ‘high’ for the relative 
deprivation of its students, but has a WP Factor below 1.025 (the boundary line used by the Council 
to benchmark for high WP Factor colleges). 
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The sections of the report that follow set out the findings arising from Stage 2 of the 
project under the same broad headings that were used to group the questions asked 
by the Agency’s consultants during their visits to the 10 colleges concerned. These 
are: 

• background, mission and ethos 

• strategies for maintaining and raising achievement 

• student recruitment, placement and induction 

• curriculum development 

• tutorial systems and other support systems 

• teaching and pedagogy 

• monitoring, evaluation and follow-up. 

A summary of key characteristics of effective performance is provided at the end of 
each section. 



4 BACKGROUND, MISSION AND ETHOS 

Catchment area and student profile 

All of the 10 colleges involved in Stage 2 of the project have WP Factors above the 
median for the sector. Though those with the lower rates of achievement tended to 
be associated with more pronounced indicators of deprivation, the large majority of 
interviewees were conscious of the challenges presented by the nature of the areas 
served by their colleges. Only in the case of College H did managers and staff 
consider that the population they served was not particularly poor. 

There were a number of references to school-leaving attainment rates that were 
below the national average, and this is true of every LEA in which the sample 
colleges were situated. There is no sign of any general connection between LEA 
average GCSE points scores and the achievement rates of the colleges concerned. 
Colleges F and G, for example, are both located in cities with similar school 
attainment figures well below the national average, but differ markedly in their own 
achievement rates. There is some indication, though, that the colleges with the lower 
achievement rates tended to be associated with a more competitive educational 
environment, judged on the basis of the relative number of other FE-sector 
institutions with overlapping catchment areas. They may therefore have encountered 
greater difficulties in attracting the most able and best-motivated students. 

As well as higher levels of deprivation, the other feature that most distinguishes the 
colleges in connection to their relative achievement rates is the ethnic profile of the 
student bodies. It is notable, for instance, that the two Stage 2 colleges with the 
lowest achievement rates (Colleges E and I) have high proportions of students 
drawn from minority ethnic groups. In the case of the College E, 65% of full-time 
students are drawn from non-white ethnic groups. Some 50 different mother tongues 
are spoken by the 15% of the college’s students whose first language is not English. 
At College I, over 40% of FEFC-funded students are from Black and Asian minority 
groups. As we have noted in the findings from Stage 1 of the project, in general, 
differences in the ethnic mix of student bodies do not appear to be able to account 
for the bulk of the inter-institutional achievement gap. Within the Stage 2 sample of 
colleges. College C, for example, also has a high proportion of students drawn from 
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minority ethnic groups and yet in 1998-99 had an achievement rate above the 
average for the 10 colleges involved. 

Some of the feedback received in response to the report on Stage 1 of the project 
hypothesised that the minority ethnic mix of the student body interacts with 
institutional factors in ways that have a negative effect on achievement over and 
above that relating to relative differences in ISR profiles. The impact of racism and 
negative stereotyping was claimed to add to the pressures associated with 
deprivation so as to place greater demands on the colleges most concerned, and 
reduce their achievement rates as a consequence. However, there is also a contrary 
argument that where the minority ethnic presence within student bodies is very 
small, their members may suffer from just as much racism and negative 
stereotyping, and have fewer peer group support mechanisms to help them cope. At 
present in the college sector, robust statistical evidence for or against these contrary 
positions is lacking. 



Mission and ethos 

All of the Stage 2 colleges are committed to widening participation and enabling all 
students to realise their full potential. In some cases - Colleges B and D, for instance 
- widening participation had been an overtly high priority for some time; in others 
(Colleges E, F and J) there was a more recent increased emphasis on serving the 
wider community, connected in part to the mergers that had formed these 
institutions. 

As we have noted, there was no obvious evidence that relative success in widening 
participation has impinged on the achievement rates of the colleges concerned in 
any consistent way. Nor did there appear to be any obvious correlation between their 
achievement rates and the relative priority accorded to widening participation. 
Nonetheless, interviewees at College A felt that they had in the past enrolled 
students onto inappropriate courses in their ‘enthusiasm to widen participation’. The 
principal of College H indicated that it was not the aim to cover every curriculum area 
or try to meet every need, and that students would be recommended to go 
elsewhere if it was more appropriate for them to do so. At this and other colleges 
there had been adjustments to the curriculum offer to match it more closely to the 
needs of actual and potential students (see also section 7 below). 

In most of the colleges, a number of interviewees mentioned attitudes and values 
that helped to inhibit attempts to raise achievement. Below average attainment in 
local schools was carried forward in the form of low aspirations on the part of 
students, which in turn affected the expectations of staff (senior managers at 
Colleges C and G referred to ‘collusion in under-achievement’ on the part of some 
staff). 

Among many of the managers, there was a raised consciousness that questioned 
‘soft’ and unfocused student-centred approaches, and instead adopted ‘tough love’ 
policies targeted at individual circumstances. At College C, for instance, they were 
challenging a deep-seated attitude summed up as ‘students matter; systems and 
data don’t’, which had inhibited a thorough examination of any problems relating to 
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raising student achievement. Interviewees there saw success in changing this 
attitude as a crucial factor in the improvements in achievement that had occurred in 
the last two years. 



Redefinition of mission after merger 

College E merged with another in 1998 and adopted a new 'community college’ identity for the start 
of the 2000 academic year, with a public launch programme. The merger was aimed, among other 
things, at a more effective coordination of a wide range of disparate community-based provision. The 
college maintained strong links with the local authority and this had led to the establishment of a 
strategic partnership that sought to address the demand for 'high quality skills, training, qualifications 
and career development programmes in the sport and leisure business'. 

College F was established in 1996 as a result of a merger between two well-established FE 
institutions. The college had subsequently merged with a third specialist FE institution. The college 
operated on five main sites, including a recently opened city-centre building. The college had 
developed a stronger community-orientated focus drawing on the strengths of the former individual 
institutions in order to address low participation in FE across the city. As a result, there had been 
growth in provision, particularly at Levels 1, 2 and Entry. Partly as a consequence of this growth and a 
perception that many students were coming from schools without basic literacy and numeracy skills, 
there had been a substantial growth in the number of additional support units claimed. 



Centrality of retention and achievement 

In the majority of the colleges, securing high levels of student retention and 
achievement had high priority within the current strategic plan. College A also gave 
prominence to widening participation, while the strategic plan of College D stressed 
growth, diversification and access. At College B there was no overt reference to the 
maintenance and improvement of retention and achievement rates within the 
strategic plan, but these issues were emphasised within the annual course review 
process. 

In many cases, the strategic importance accorded to improving student achievement 
had been stimulated by FEFC Inspection reports. Generally speaking, the second 
round of Inspections was much more explicit than the first about the need to improve 
student achievements, in some programme areas and courses, if not generally. 
Colleges with some of the higher achievement rates within the sample did not 
escape these strictures. Thus, at its last Inspection in February 1999, College B was 
directed to ‘address the declining retention and pass rates on some courses’. The 
major drive to raise achievement at College C was stimulated by a 1997 Inspection 
report that led subsequently to the acquisition of significant financial support from the 
Standards Fund. Ironically if unsurprisingly, though, the colleges with the lower 
achievement rates were among those now according the highest priority to the issue. 
At College E, for instance, poor retention and achievement was highlighted in the 
reports of Inspections taking place in 1994 and 1998, and was addressed further in 
the re-inspections of the IT programme area in March and November 2000. 

A combination of factors seemed to characterise the colleges with the higher levels 
of achievement, or the most notable sustained improvements: 
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• the centrality of student achievement within the mission of the college, coupled 
with a widespread awareness of this across the staff 

• a belief among the majority of staff that interventions at college-level could make 
a difference, and that students were not predestined for low achievement 
because of their backgrounds 

• a general commitment to quality assurance and improvement across course 
teams and individual teaching staff, as well as at managerial level. 

Similar attributes have already been noted in studies of effective college self- 
assessment (Dixon and Walker 2000). The ability of college MIS systems to produce 
meaningful and timely data was likewise an important contributory factor. Notable 
improvements in this area at College C, for instance, were seen by interviewees 
there as having been central to improved achievements in recent years. Regular, 
good quality data also helped governing bodies to monitor the academic 
performance of the college effectively, an area that until recently came in for regular 
criticism from the Inspectorate (FEFC 1998, 1999). 

Some colleges appeared to have tackled the issue thoroughly, over a longer period, 
and to have been more successful in embedding a culture of continual improvement 
at course team level. Having said this, colleges where achievement rates remained 
well below the national average in 1998-99 - Colleges E and I, for instance - were 
now making major sustained efforts to address the situation. As we have noted, 
trends nationally, as well as within the sample of colleges, indicate a steady closing 
of the ‘achievement gap’, which suggests that these initiatives are having the desired 
impact. 
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strategic initiatives to raise student achievement 

At College I, the draft strategic plan for 2000/03 made a number of references to raising 
achievement. Its first priority objective was ‘further substantial improvements in student retention and 
achievement'. Associated tasks included the identification of courses for withdrawal, modification or 
replacement in the following year. The second priority was ‘improving the processes of teaching and 
learning'. Supporting initiatives included a revised staff development programme for all teachers, 
focusing on how students learn. Raising achievement rates was a major focus in 1999—2000, the 
college having failed to meet or exceed national benchmark figures for all except level H (4 and 
above) long qualifications for the 19+ age group. Provisional statistics for 1999—2000 showed 
significant progress. The deputy principal was developing a ‘raising retention and achievement 
project’, which will set out in detail the processes being developed by the college, with dates for 
implementation and identification of those responsible. The self-assessment report aimed to ‘promote 
and celebrate students success' and referred to the initiation of a ‘culture of performance review and 
monitoring' with explicit lines of accountability and increased ownership of targets. One of the 
responsibilities of the deputy principal was raising quality and achievement. She had been in post for 
a year. A clear cross-college vision had been developed with the involvement of governors. The aim 
was for an overall improvement in achievement figures of 17%, a deliberately ambitious target to 'kick 
start' the process. Key features of the approach were; 

• to raise awareness of the importance of student success 

• to identify problems in order to find solutions 

• the avoidance of a blame culture 

• a quality improvement strategy in the strategic plan, with governor involvement 

• the development of consistency across the college. 

National standards were taken seriously. Working to achieve Investors in People was seen as a 
positive process. The college will be working towards the European Quality Standard. There was a 
perception among middle managers that the emphasis was shifting from student numbers and 
funding units to a focus on the quality of provision and student achievement. There was a college- 
wide Inclusive Learning Action Plan, which was cross-referenced with the College Strategic Plan, the 
College Staff Development Plan and a range of initiatives relating to widening participation. 

At College J, strategic emphasis was placed on a culture of continuing improvement. The first 
strategic priority in the 1999—2000 Strategic Plan was for the college 'to continuously improve its 
service, particularly in its core business of teaching and learning and to raise the level of student 
achievement'. This would remain a priority in coming years. In a section on 'Consultation with TECs 
and Lifelong Learning Partnership', under Retention and Achievement the Plan stressed; 

• ‘the quality and appropriateness of provision and the importance of participation being part of a 
lifelong learning programme, not just short-term recruitment' 

» ‘balancing the widening participation agenda with genuine improvements in the quality of student 
experience, so we can continue to raise achievement levels'. 

All of these intentions were linked to objectives in the strategic plan and targets in operational plans. 
They were also reflected in the development of a range of policies and the formation of a number of 
groups designed to improve quality assurance. 
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5 STRATEGIES FOR MAINTAINING AND RAISING 
ACHIEVEMENT 

College-wide strategies 

Though all colleges had in place systems for the review of and accountability for 
student achievement, in Colleges B, D and F there was far more autonomy at 
departmental level in the improvement strategies that were adopted and 
implemented. In the latter two cases, in particular, this reflected the natures of their 
organisational structures. Though there were some benefits in devolved structures in 
encouraging ownership of issues, there were also inevitable inconsistencies in the 
extent to which college-wide strategies were evolved and implemented. At College F, 
for example, there were some notable faculty-wide retention and achievement 
strategies, but senior managers acknowledged the need to develop a more coherent 
planning cycle, to increase the effective monitoring of college-wide strategies. 

In recent years, major college-wide initiatives to improve student retention and 
achievement had been put in place, or reformed and augmented, at most of the 
colleges. In some cases, as at Colleges E and F, SMTs recognised that in the past 
cross-college planning had been fragmented and that ‘too many loops were left 
open’. There was a need to focus on systems to ensure greater consistency and to 
align the various planning and review cycles. 

There was a range of distinctive features about each college’s approaches. 
Nevertheless, a number of common aspects emerged; 

• regular review in conjunction with curriculum leaders and course teams 

• action planning, targeted at courses or programme areas giving cause for 
concern 

• clear accountability and reporting in the way in which action points had been 
fulfilled, involving regular face-to-face meetings between managers and 
individual staff with a major focus on retention and achievement 

• related activities to review the quality of teaching and tutoring, and to provide 
associated staff development. 

Once again, though, it should be emphasised that, in isolation and imposed top- 
down, such initiatives by no means guaranteed success, however well designed the 
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structures and systems. Culture shifts also needed to occur, so that a critical mass of 
staff accepted that improvements in student achievement were by no means beyond 
their influence, and that things would not change without their commitment to raising 
achievement. Such changes took time to embed. Generally speaking, and making 
due allowance for differences in their ISR profiles, the higher-achieving colleges had 
been relatively more successful in making this transition. 



Recruitment, diagnosis and support 

Coiiege A was carrying out a range of strategies to improve its achievement. These included: 

• improving initial assessment and recruitment procedures 

• providing appropriate support for the significant number of students with poor levels of basic 
skills 

• focusing on evidence of students’ attitude and motivation towards achieving prior to enrolling on 
college courses. 



Standards Fund and other initiatives 

At Coiiege C achievement rates were now a major concern for everyone. This was not always the 
case. The FEFC Inspection report in 1997 indicated that achievement rates did not figure in the self- 
assessment reports. The college was identified as ‘a college causing concern’ by the FEFC and so 
got considerable Standards Fund help; 

• £64K for improving part-time achievements and to review management and quality systems for 
hourly paid staff in particular 

• a new appointment oversaw data capture and reporting for ESOL; new ESOL qualification aims 
were developed 

• £61 K for improving tutorials; the 1 997 Inspection report had found these to be inconsistent; there 
was now a new tutorial policy and tutorial handbook and there were now seven lead tutors 
answerable to the quality manager 

• £1 1 8K to improve data capture and reporting; MIS data was put on the intranet in a very 
comprehensive and user-friendly format, which allowed staff to update records. 

These Standards Fund initiatives were being continued this year, with the staff appointed being kept 
on. There is a general feeling that all the Standards Fund initiatives were very productive and useful. 
Non-Standards Fund college-wide initiatives included; 

• Achievement reviews: continual monitoring of retention and achievement up to Executive level. 
There were achievement reviews for heads of schools and for course managers. These had 
three-year trends and compare performance to national benchmarks. Achievement reviews 
occurred every term, thus monitoring ongoing student attendance and progress. Areas for action 
were identified and followed up. The quality manager reported back to the Executive on whether 
these action points had been achieved. 

• Business Review Meetings: held about 1 -2 times a term, these looked at recruitment, budget, 
and so on. 

• Quality panels for each school in the college; each one included a lead tutor, an additional 
learning support person, a key skills person and a practitioner. Remission was allowed for this 
role, which was central. 

• Lead tutors: they were responsible for the quality of tutorials - a substantial role including training 
staff and observing whole class and individual tutorials. 

• Achievement action plans: these were required for at-risk courses. 

• Level 2 emphasis: prestige centres had been created with an excellent environment specifically 
for Level 2 courses. The hope was that this would raise the morale of both students and staff at 
Level 2. 



Course-level initiatives focusing on the student 

At College D, bottom-up and top-down approaches were used for the development of strategies. 

Each school or department was fairly autonomous, though accountable to the SMT. More local 

strategies of interest included; 

• Breaking assignments down into tasks: one course team was setting one task per week rather 
than whole assignments. This evened out student workload. 

• Regular testing: the section leader for Maths and IT in the college's sixth form centre had 
attention and achievement figures well above national benchmarks. Every six weeks there was a 
timed assessment or tests written by the class teacher. The bottom 5% to 10% of students was 
required to do extra work and/or come for extra help. Attendance for tests was good. 

• Parental involvement: if a student was not present to collect work it was posted back without 
using the college franking machine, so that students could not easily identify it and conceal it 
from parents. The college had also tried marking the envelope 'attendance alert' so parents were 
aware there was a difficulty. Parents were written to if there was an attendance problem. 

• Spot checks: attendance was checked by the heads of departments from time to time without 
warning, and letters sent out to parents (without the college frank on it) if a student was absent. 
This worked well as students never knew when the check would be and they could not intercept 
the letter on the doormat. As a consequence of these measures a culture of attendance had 
been created. 

• Tough but supportive ethos: there was confidence building 'pep talking' and motivational input 
from the head of department, to encourage student self-help. 



Prioritisation of specific strategies 

At College G, the principal, assistant principal and heads of curriculum were focusing on the following 
specific strategies: 

• introductory courses for young people and adults 

• a learning policy for 16-18 year olds 

• a coherent curriculum offer 

• issues of disaffection and their remedies; white male exclusion was a particular problem in one 
area of the city 

• basic skills and key skills initial assessment and support 

• integrated student support and standards for non-attendance 

• retention and achievement based on neighbourhoods and not on global figures; retention and 
achievement was seen as needing to relate to student background. 

Two issues still caused problems, and had not been fully resolved: 

• tension between the need to recruit for funding units and the need to improve retention and 
achievement 

• the absence of value-added measures that could be applied across the whole curriculum. 



Focus on point of delivery 

On formation as a result of merger. College J was restructured into programme-based teams, with 
the intention to 'put resources and responsibility as close to the point of delivery, and therefore as 
close to the students, as possible'. An Achievement Team of staff with key management 
responsibilities was set up to implement the strategy. The MOT ('Meeting Our Targets') group was 
established. Senior staff scrutinised targets and achievements at course level with course leaders, 
who had to design and implement a recovery strategy for failing or unviably small courses. If 
unsuccessful, courses were withdrawn. This process was viewed positively by most teaching staff. A 
number of significant changes to the curriculum and improved success rates had resulted. The 
process was continual. The MOT was backed up by service level agreements that defined agreed 
actions and specified completion times. Staff development in target setting had also been provided. 
National quality standards were taken seriously. IIP, The Charter Mark and ISO 9000 had been 
achieved through building them round the course team profile. The college had used the Standards 
Fund to invest in training and support for part-time staff. 
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Communication of strategies 



Teaching staff at all the colleges, as well as their managers, were in no doubt about 
the importance of student retention and achievement. There were some differences, 
though, in attitudes to the way in which these issues were communicated. 

Curriculum managers at College H, for example, felt that the college’s mission and 
ethos was communicated effectively, but that not all staff took on the necessary 
individual responsibility for tackling student achievement. Some of the teaching staff 
there, however, indicated that though they were kept informed about retention and 
achievement the ‘message’ often appeared negative and not encouraging to staff 
involvement. 

Generally speaking, colleges dealt regularly with retention and achievement at staff 
meetings, in newsletters and via staff development activities. There was a wide 
range of examples to raise staff awareness of the issues, to disseminate evidence 
from national research in the field, to encourage effective practice and to improve 
teaching quality. 

Improvements in the timeliness and quality of MIS data had clearly made a major 
impact in raising awareness of the issues, and in convincing at least some staff of 
the need to address them. There were a number of instances where hitherto staff 
had not realised how their own programme areas and courses lined up against the 
national average, and with comparable provision in their own institutions. Good 
quality data, communicated effectively within a culture of continual improvement 
rather than apportionment of blame, made a key difference in ensuring that staff 
confronted achievement-related problems, rather than concealed or denied them. 



Learning development 

College F had recently devoted a one-day staff conference to retention and achievement issues. The 
extensive development and provision of learning support (termed ‘learning development’) was a major 
strategic response to improving retention and achievement and is perceived as a significant strength. 
Senior staff pointed to the use of the learning support strategy as a springboard for the development 
and implementation of other teaching and learning strategies, for example analysis and use of 
preferred learning styles information. 



Staff meetings and learning advisers 

At College I, centre-based meetings and termly full staff meetings with the principal or deputy 
principal had been established, with a view to raising awareness among staff. The importance of 
retention and achievement was stressed and attendant issues explored. A team of 'lead learning 
advisers’ was established from Spring 2000. The aim was for the advisers to support the tutorial 
process, with a particular focus on appropriate course placement and transfer. Advisers would also 
have a role in monitoring tutorial provision and the learning review process. 



Celebrating successes 

College J had developed a Student Retention and Achievement Strategy, drawing on published 
research. This committed the college to the development of learner autonomy, the publication and 
analysis of achievement data, the refinement of Student Services and the development of appropriate 
curricula. ’Success centres’ were established, publicising results and individual student achievements, 
at various levels. These might include finding work, getting into university or completing successful 
projects. The college held an annual prizegiving ceremony. 
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Benchmarks and targets 



All colleges are required by FEFC to set targets for student retention and 
achievement making use of nationally published benchmarking data. The colleges 
involved in this project mostly used the figures for the high WP Factor colleges as 
comparators. There were relatively few examples of attempts to define more precise 
benchmarks, though a number of interviewees were conscious that the national 
medians were not necessarily appropriate to their own circumstances. At College A, 
the SMT rightly noted that their student profile was atypical of that of the majority of 
sixth form colleges. High WP Factor FErtertiary college data was therefore also 
taken into account. At College H, the adult Basic Education area benchmarked itself 
against equivalent provision in a neighbouring college. 

The benchmarking process inevitably meant that colleges with a trend of 
achievement well below the published medians had the most demanding targets. At 
College I, therefore, a 17 percentage point uplift was aimed for by the end of the 
current academic year. 

Quite commonly, internal benchmarks were also set that triggered remedial 
attention. At College H an action plan was required for any course with an 
achievement rate that fell below 50%. Persistently low achieving courses had been 
withdrawn from the curriculum offer altogether at Colleges C, D, G and J. 

Here again, relevant data was seen as having a crucial part to play. At College E 
improvements in the timeliness and accuracy of MIS data were seen as a matter of 
urgency. Systems to speed up the collection, input and analysis of data were under 
development. Linked to these were new arrangements for planning, review and 
target setting that were expected to improve the college’s ability to tackle retention 
and achievement as they came on-stream. 

Ideally the benchmarking process should result in the setting of challenging but 
realistic targets. Differences in the student profiles of colleges clearly influence the 
achievement rates that they can reasonably be expected to record. Nonetheless, the 
findings arising from Stage 1 of this project suggest that all High WP Factor colleges 
should be capable of delivering rates of student achievement that are above the 
current median level for this group. In that sense, therefore, the national medians are 
not ‘unfair’ as target indicators for those institutions with achievement rates that fall 
well below the current median level for their group. 



Online information 

At College C, a bar graph comparison of results was made available via the college’s intranet, 
incorporating benchmarks for retention and achievement and showing the three-year trend. These 
had to be used for reviews and course team reports. They were available online to every member of 
staff for every course in a user-friendly format. The senior curriculum manager stressed the need for 
every target to have an accompanying strategy for its achievement. 
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Achievement benchmarking and action planning 

At College H, internal benchmarks were based on national benchmarks (where available) and what 
the college considered reasonable achievement'. Targets for retention and achievement were now 
set through 'achievement benchmarking'. MIS supplied achievement data to curriculum managers. All 
courses with less than 50% achievement had to draw up an action plan, which was now monitored by 
the curriculum standards manager. 



Key characteristics - caiiege-wi(h approaches 

• annual course review and self-assessment procedures, with the active 

involvement of curriculum leaders and course teams, designed to identify 
strengths to be built upon and weaknesses to be addressed ■ 

• action plans arising from the reviews, containing strategies to tackle wealmesses ^ 
and meet targets 

• regular monitoring of retention and achievement in relation to action plans, and 

iiyilcieat:accduhtabiiity:iihdi;ti|iiii|ibi(liliiiwu|iiiiii;is^ 

■ ■■ a feed-through fromdourse revteW:lntd;teachihg- and tutonaiyj|i®|HiB 
associated programmes of staff development 

• an expectation and encouragement of bottom-up as well as top-dovi^1hi|||BS^ 
to improve the design and delivery df- the curriculum' ' ■ ' 

• secunng parental involvement . 

• recognition that there is no'one foolproof way to raise; 

y:yy:yyyyyy;ytii|eyyofyiiirati|iiiyiii:y|i;yH:y|||||iiiyii|| 



6 STUDENT RECRUITMENT, PLACEMENT AND INDUCTION 

Pre-enrolment information 

There is now a considerable body of evidence that if students are supported 
effectively to select courses that are appropriate to their needs, if they understand 
fully what the courses will entail, and if they are otherwise assisted in ‘settling-in’, 
then their chances of eventual successful completion are notably increased 
(Martinez 1997; Martinez and Munday 1998; Martinez 2000; Comptroller and Auditor 
General 2001 ). The college management teams were certainly well aware of the 
importance of enrolment and induction programmes in reaching retention and 
achievement targets. 

The first stage in this process - the communication of pre-enrolment information - 
was being strengthened at a number of the colleges concerned. At Colleges F and I, 
the introduction of a centralised admissions system had improved the quality of the 
way p re entry enquiries were handled. In addition to the standard prospectuses and 
course leaflets, details of all courses had been placed on the website at College C, 
and on a widely available CD-ROM at College D. 





There were a number of staff and student interviewees, however, who felt that pre- 
course information still varied considerably in quality. Ironically, information was far 
more successful for prospective students who already had a reasonably clear idea of 
the course and qualification on which they wanted to enrol. Those who were least 
sure of what they wanted were the most likely subsequently to feel that they had not 
realised what would be involved and that they had made the wrong choice. Students 
at College G were also critical of the national call-line learndirect, which was viewed 
as being run by non-professionals unable to provide specialist advice in the level of 
detail that was needed. The problem of combining a readily available enquiry service 
with the capability of providing detailed and specific information was one that 
exercised staff at most of the colleges that were visited. College F’s Student 
Services section had attempted one solution in the form of a Frequently Asked 
Questions guide to studying at college aimed at 16-19 year olds. At College E, 
programme-specific brochures supplemented the main prospectus - for example, 
the Leisure College prospectus, which featured details of the college’s sport, 
recreation and leisure provision. 

It would appear, though, that initial information and guidance is an area that colleges 
must work on if they are to maintain their position. In general, success in widening 
participation increases the proportion of entrants who are uncertain of the direction 
that would be best for them to take. Such new entrants are also among the least 
likely to access information via electronic media. Getting the right quantity and 
quality of information across to individuals who may well not know the questions that 
they should ask remains a key challenge for colleges - and one that is unlikely to be 
met without a significant amount of face-to-face mediation. 



Placing students on the right course 

Management teams at a number of the colleges regarded more effective placement 
onto courses as a key issue in maintaining and improving achievement. Processes 
of enrolment interviewing, diagnostic assessment and course placement had been 
strengthened in several cases, underpinned by staff development for those involved. 

There were a number of problems in this area that were difficult to overcome. 
Effective guidance interviews are time-consuming, and require staff skills that are by 
no means universal. A feature of a number of the systems that were observed was 
the attempt to identify ‘at risk’ students so that learning and other support services 
could be alerted. Several teaching staff who were interviewed, however, identified 
student motivation as being a more important factor than personal circumstances. 
Examples were quoted of students from the most deprived backgrounds who 
succeeded against the odds because they were strongly focused on an ultimate goal 
that required their successful completion of the course in question. Opinions differed 
as to the extent such motivation could be fostered by the college. At College E, some 
teaching staff noted that information about students gathered by admissions staff 
was not always passed on to tutors, with a consequence that learning support needs 
were not always diagnosed early enough, if at all. College B contrasted with College 
A in having a more developed college-wide system of course placement. Though 
recruitment procedures at the latter had been tightened, the ethos was noticeably 
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more inclined towards 'open access’ in line with the high priority accorded to 

widening participation. 

Characteristics of the most effective systems of course placement appeared to be: 

• well-developed advice and guidance services to prospective students 

• rigorous enrolment interviews, supported by clear college-wide procedures 

• regular review of course placement in relation to the curriculum offer and pre- 
enrolment information, to ensure an appropriate choice of clear 'learning 
pathways’ from foundation level onwards 

• diagnosis of 'at risk’ students, to identify and alert the need for subsequent 
learning and other support 

• the availability of 'taster’ sessions and the ability, in the event of dissatisfaction 
with the course chosen initially, to make course transfers at an early stage 

• careful selection of staff involved in process, supported by tailored staff- 
development programmes in associated skills. 



‘Joined-up’ recruitment and placement procedures 

At College B, there was a well-developed process to ensure that students were placed on the most 
appropriate courses, achieved by use of common interview forms and identified staff who carried out 
interviews following a training session from the Student Services officer. Course entry guidelines were 
provided for each interviewer and trained college careers guidance personnel were available to 
support the initial guidance interview. If a student reference identified ‘at risk’ criteria - for example, 
poor time management or attendance at school - then use of a faculty contract was suggested. Each 
student interviewed had to apply for the course (even though they might already have followed a 
course at the college) and common documentation was used. The college had a team of trained staff 
(ESOL, basic skills, dyslexia, and so on) who provided the ‘Skills Development Service’. It was 
recognised that students following courses at all levels might have learning support needs, either at 
induction or during the course. 



Centralising admissions 

College F had recently implemented a centralised admissions system as a direct response to 
problems of drop-out resulting from mismatches between student expectations and the requirements 
of their intended programme. The admissions process delivered a broader entitlement to advice and 
guidance enabling students to gather the maximum information about their intended programme and 
the college. The admissions system used interviewing tutors and there were plans to accredit or 
’badge’ them in the future as part of the quality assurance process. Programme areas conducted 
more detailed subject specific interviews. Some faculties - for example Hair and Beauty - strongly 
encouraged the involvement of parents or other significant adults in the interview process. 



Induction 

Students at all of the colleges were provided with some kind of induction programme 
aimed at helping them orientate themselves with college and course. At College B 
there was a ‘whole-college’ approach to induction, which took place during the first 
three weeks. All staff there went through a training day to ensure a consistent 
approach and receive up-to-date information. At College C ‘induction checklists’ 
were provided in a handbook for tutors. Student evaluations of the induction 



programme were built into the course review process. At College F, students were 
increasingly enrolled onto generic ‘pathways’ programmes with final choice of course 
made after a follow-up advice session approximately six weeks into the programme. 
College C had successfully introduced a mentoring system for black students, 
involving effective role models from the community who volunteered for the post. 

Most staff interviewees felt that the right balance had now been arrived at between 
general induction to the college, and the specific requirements of a particular course. 
Most colleges also used induction as an opportunity for a more thorough diagnostic 
assessment of learning support needs. Several interviewees remarked that despite 
the attention to pre-course information it was only at this stage that many students 
began to realise the commitments that would be involved in order to complete their 
chosen programmes successfully. There were a number of references to an 
increasing emphasis on student obligations as well as upon their entitlements. 

Students interviewed generally found formal induction programmes useful. Younger 
students found that there were differences in teaching and learning styles to those 
with which they had been familiar at school. For many adults, college represented a 
return to learning after a long break, and they therefore welcomed help in 
familiarising themselves with what would be required. A number of student 
interviewees regarded early-stage course transfers as being rather more difficult to 
arrange than was (in theory) the case. 

Despite a generally positive reaction to the induction process, it was striking how 
much students’ early experiences of the course proper coloured their attitudes 
towards it. Only two months after initial enrolment, many student interviewees had 
developed very firm views on the kinds of demotivating factors that made them 
contemplate changing course or leaving college. A number of these resulted from 
external pressures, including: 

• peer pressure - sexism, bullying, social distractions 

• financial pressures - some students in part-time employment were tempted to 
extend their hours of work 

• health problems 

• family commitments 

• travel-to-college time. 

Other demotivating factors were more directly within the influence of the college: 

• concentration by teaching staff on ’bright’ students at expense of others 

• quality of some teaching - unprepared lessons, weak voice, apparent lack of 
enthusiasm for subject, dull lessons, unexplained jargon, negative comments 
about students’ abilities or where they might end up in life 

• perceived lack of relevance of key skills 

• negative learning - inability to complete a task or understand a concept, a sense 
of having failed the same subject already, falling behind with work 

• promises unfulfilled -course content not what was expected, unsatisfactory 
equipment or facilities, lack of work-experience, perceived threats to prospects of 
progression 

• long college day or timing of sessions. 
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Induction procedures and ‘taster’ sessions 

At Coiiege B, students unsure about which course to choose were enabled to access 'taster' 
sessions over 2-3 weeks and could also arrange one-to-one or small group discussions, or 
information and support sessions with subject teachers to ensure they understood what different 
courses would entail. All students undertook a range of initial assessment activities during induction. 
Information from initial assessments was given to the student and to heads of department for tutors 
and teachers, as well as being used to identify students requiring individual learning support. This was 
discussed with the student and a joint agreement was signed. The Skills Development Service also 
responded to referrals by tutors or students, and agreed individual education plans with students - a 
commitment of 3.5 FTE staff, with the college seeking to recruit more staff to this section, which was 
seen as having a key role to play in improving achievement and retention through the provision of 
ESOL and communication skills support to a large number of students from minority ethnic groups. 



Generic pathways programmes 

At College F, students were increasingly enrolled onto generic 'pathways’ programmes with final 
choice of course made after a follow-up advice session approximately six weeks into the programme. 
The pathways programme covered a broad range of induction activities and allowed for diagnostic 
assessment of basic skills and the identification of possible risk factors including behavioural 
problems, punctuality and attendance. 



Readiness to learn 

At College J, initial assessment included a focus on students' 'readiness to learn' - the extent to 
which they had realistic goals and the confidence and skills to work at their chosen level. There was 
an attempt to replace the stigma associated with learner support with a culture of entitlement. The aim 
was the early recognition, availability and provision of additional support. The tutorial system made a 
key contribution here. Learning support was well resourced and well integrated. A team of staff was 
established with a 0.5 FTE member in each vocational area. Staff had gained qualifications through 
professional development arrangements. Staff testified to the positive impact of increased resources, 
particularly for childcare. 
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7 CURRICULUM DEVELOPMENT 

Design of curriculum 

Across the colleges there was widespread evidence of increased attention to 
reviewing and adjusting the curriculum offer to match the needs of prospective 
students, and to help sustain their retention and achievement once enrolled. 
Sometimes, as at College E, there was a realisation that not all students had been 
able to access the curriculum at an appropriate level. As a consequence students 
had been enrolled onto GNVQ advanced courses, who were then unable to cope. 
This situation was now being rectified. In other instances, further progress had been 
made. At College B ‘traditional’ A-level staff were said to be increasingly accepting 
the need to diversify and to provide more lower level courses and more vocational 
alternatives. This change of attitude had been helped by the growing numbers of 
students who had progressed upwards through the levels during their time at 
college. Colleges C and D both now had wide offers covering all levels from pre- 
entry to Level 4 and above, so that true ‘curriculum ladders’ were in place. 



Curriculum innovation and withdrawal 

There was a noticeable trend among the general FE/tertiary colleges to strengthen 
vocational provision and to reduce the number of courses leading to academic 
qualifications. The aim was to adapt the curriculum to the needs of learners and 
improve their achievement. GCSEs were being phased out at Colleges I and J, and 
A-level provision rationalised at College G. 
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In other cases (such as Colleges C and D) awarding bodies had been changed with 
a positive impact on achievement. At College J, modular and short courses were 
being developed to give students an early experience of success and incremental 
achievement. All full-time courses now led to more than one qualification. Moves 
were also underway towards the development of a unitised curriculum, which could 
offer credit accumulation. Basic skills provision had been integrated in mainstream 
courses, though discrete basic skills courses with vocational links were also 
available. 

At College F, the development and provision of learning support (known there as 
‘learning development’) was a major strategic response to raising retention, and 
thereby achievement. Learning support was viewed at the college as a 
developmental process delivered within a vocational context. The link to the 
programmes of the individual students concerned was seen as essential, and as 
accounting for the good level of uptake of learning support in some faculties. 
Generally noticeable also was a willingness on the part of management teams to 
close courses that had a history of persistently poor retention and achievement, in 
spite of attempts to improve their position. This happened even where the level of 
demand was in theory sufficient for them to be maintained within the curriculum. 



Curriculum change to support student achievement 
College G: 

• development of a more vocational curriculum 

• development of courses to ensure progression opportunities between levels 

• introduction of additional GNVQ courses at intermediate level 

• segmentation into client groups 

• introduction of summer schools, 'taster's and induction days 

• ethos of the ‘creative curriculum', which utilises real experience and better subject coordination 
across college 

• concentration on basic skills and key skills support 

• rigorous tutorial system and individual action planning. 

College H: 

• move away from academic to vocational curriculum 

• development of sporting academy 

• cross-college key skills 

• strengthening college tutorial system and adoption of Greenhead value-added model where 
appropriate 

• childcare courses run as one group for six weeks and then split into NNEB or BTEC 

• Basic Education students enrol on a generic course and then decide on achievement 
qualification at a later date 

• construction students are screened into levels 

• areas set targets to improve vertical progression 

• implementation of Curriculum 2000 and enrichment programme. 
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Curriculum structures, organisation and delivery 

A number of more flexible approaches to timetabling were observed, to allow 
students to tailor their studies more easily to their other commitments. For example, 
in College F the Faculty of Administration and Business Technology operated on a 
48 week year in order to allow scope for varied attendance patterns and rolling 
starting dates. Rolling starting dates were also a feature of some provision within the 
Faculty of Hair and Beauty Therapy. Timetabling there had also eliminated ‘slack 
time’ and start times had been moved to 09.30 to accommodate childcare 
commitments. 

Management staff were generally more conscious than teachers of the concept of 
inclusive learning, differentiating the needs of individual students and adapting to 
them appropriately. Here, the role of FEFC’s Inclusive Learning Quality initiative was 
acknowledged. However, it was also widely recognised that there was still 
considerable potential for progress, and inclusive learning was featuring increasingly 
within programmes of staff development. 



Expectations of students and staff 

A general feature of the courses and programme areas which enjoyed the greatest 
success in improving or maintaining achievement was the high level of expectations 
of both students and staff. In these cases, a culture was apparent that combined 
high degrees of accountability and ownership. Staff who had high aspirations for 
themselves were frequently seen to lead by example with their own students. Within 
a college-wide framework of policy and support, staff who felt encouraged to do so 
could be highly innovative in improving the curriculum. Some of the most striking 
examples of improvements in curriculum delivery, which had been seen to have 
dramatic positive impact on retention and achievement, had been undertaken on the 
initiative of main grade lecturing staff. Because such individuals had aspirations to 
improve a problematical situation, and because they did not feel powerless to act, or 
that remedial action was someone else’s responsibility, they were able to bring to 
bear their specialist knowledge of both subject area and students to great effect. In 
the absence of such a culture, ‘top-down’ initiatives were unlikely to succeed to 
anything like the same extent, however laudable their intent. 
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Innovations in Motor Vehicie Engineering and Sport, Leisure and Tourism 

At Coiiege E, Motor Vehicle Engineering provided a good example of how staff had worked hard to 
develop the curriculum to make it more responsive to student needs. There was an even split in time 
spent in workshops and classrooms with a good mix of activity in each. The curriculum had been 
modularised to enable students to work more effectively at their own pace, and portfolio development 
had been 'demystified' for students. Workshops had been redesigned and rebuilt to make them more 
reflective of garage practice. Motor Vehicle students spoke enthusiastically about the course. They 
felt that the mix of workshop to classroom delivery was about right even given the fact that some 
students could gain further workshop experience through their jobs while for others college provided 
the only workshop experience they could gain. The use of the final hour of the day for portfolio 
building and review was welcomed. Students also noted that tutors had high expectations of them as 
students and that there was a distinctly work-like ethos within the area. They valued the practical work 
experience that tutors possessed. 

The Sport, Leisure and Tourism area provided another example of the development of high quality 
curriculum provision. The leadership and motivational skills of the curriculum area coordinator 
appeared to be a significant factor. Provision had recently relocated to a suite of rooms at the stadium 
of a nearby Premier League football club. This was as a result of a number of strategic partnerships 
that the college had developed, notably with the club itself and with the commercial arm of the local 
authority's Leisure Service Department. Students referred to the close fit this gave between their 
learning environment, the content of their programmes and their career aspirations. There had been a 
number of early problems in accessing resources, some still to be resolved. Parts of some 
programmes were still delivered at the main site. Given such a high profile location, the tendency for 
students to be attracted by the facility itself was a distinct possibility. However, the students 
themselves specifically discounted this. Effective recruitment advice and guidance had helped to 
ensure that students were on courses because they matched their longer-term aims. Only one 
student was known to have left the course because of a misapprehension about its content. Students 
referred to the distinctive atmosphere and culture at the stadium site. Staff had high expectations of 
students and knew the students very well as individuals. Students felt that there was a greater focus 
on work and study - one student contrasted the ethos with the 'laid-back' approach that characterised 
her previous experience on a business studies course. Staff were seen as wanting students to do well 
and succeed - 'they want you to be there' and 'teachers want to do their best for you' were two 
comments made by students. Students felt that staff were 'great communicators and motivators'. They 
noted the impact that the curriculum coordinator had on their experience and how his enthusiasm was 
transmitted to other staff working within the area. 



High expectations 

At College F, curriculum delivery in a number of faculties was characterised by markedly high 
expectations of students (and staff). Staff referred to a success culture and there were elements of 
creative competition among students that were viewed as motivational. Students interviewed 
confirmed these high expectations and their motivational benefits. In vocational areas high 
expectations were linked to the development of a strong, work-related, professional ethos. In Hair and 
Beauty Therapy and Motor Vehicle Engineering, for example, students were expected to behave and 
work in exactly the same way as if they were employed in a salon or workshop. Staff in these areas 
had current industrial experience and were very much viewed as professional role models by 
students. Work experience, either real or simulated, was viewed as an essential part of the learning 
process. 
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Individual lecturer initiative 

At College G changes had been made in the Motor Vehicle Engineering curriculum delivery area, 
where the lecturer had high drop-out and low achievement prior to conducting his own review of 
curriculum delivery. Multimedia assignments had been created based on excellent CD-ROMs and 
written with PTS software (cost £18,000). The lecturer had also developed key skills assignments 
using the same software. He believed that college retention and achievement could improve radically 
with more changes in curriculum delivery along these lines. Retention and achievement in the area in 
question had improved markedly since these changes. 



““ d0Sign fhtivery of corriootum 

adaptation of curriculum portfolio in light of changes in the profile of the 
ifittiehtbodv. and movements in the labour market 

* r; a wfingness to withdraw from areas of provision with achievement rates 
; perslsterj% below target 

* liie a full *curriculum ladder* allowing progression right the way . 

I iihrough from pre~entry to Level 4 and above 

♦ :::a Widespread practical awareness of the concept of inclusive learning manifested 

differentiate approaches to the needs of individual learners 

♦ - a proactive approach to ensuring the take-up of additional learning support by 

ttrose who required it 

♦ Tiigh expectations of students, based on teaching staff who provided good 

professional rote models 

the integration of related work ex^rience or effective 
or both 'r 

• coordinated scheduling of coursework to eliminate bottlenecks 

^ review and adjustment of the timetable in relation to students' other 
commitments. 



8 TUTORIAL SYSTEMS AND OTHER SUPPORT MECHANISMS 
Types of tutorial system 

There was a widespread recognition of the central contribution that tutorial systems 
could make to improving and sustaining student retention and achievement. Tutors 
were seen as a vital personal link with individual students, able to keep a regular 
check on their academic progress and personal circumstances, and to help ensure 
that any problems were confronted and dealt with should they arise. Five of the 
colleges (Colleges C, H, E, I and J) had either recently reformed their tutorial 
arrangements, or were in the purpose of doing so, in order to strengthen the level of 
support to students. At Colleges D and F, there was considerable school and faculty 
level autonomy in tutorial arrangements, and systems and level of support varied. 

A typical feature of such reforms - as at College J - was some redefinition of the 
tutors’ role to place greater emphasis on monitoring academic progress and 
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achievement, and making early intervention in the event of absence or problems in 
completing coursework successfully. There was now more recognition of the need to 
stretch students. The tutorial curriculum was timetabled, tutorial sessions included in 
the teaching and learning lesson observation cycle, and tutor records audited. 

At College B, the students were timetabled for two hours of tutorial support each 
week. Wherever possible, the tutorial role was allocated to members of staff who 
regularly taught the students for whom they had responsibility, so that a close and 
familiar relationship was possible. Weekly reporting via an electronic registration 
system enabled tutors to check on punctuality and absence. Tutors met with 
management every fortnight to discuss operational issues and each tutor had a 
designated support manager to whom they could refer daily. At College C, tutors 
typically delivered one hour per week of group tutorials, one hour of individual 
tutorial, and half an hour of key skills tracking. 

There was a range of evidence of general improvements in the ability to detect 
students' needs for additional learning support, and to make a variety of flexible 
arrangements for its provision. Securing and sustaining take-up was a persistent 
problem, however, especially among some of those most in need. 

In part, the review and strengthening of tutorial arrangements was driven by a 
consciousness of the proportionately greater needs that became apparent when 
participation was widened successfully. Staff at College G, for example, were 
conscious of operating against a background of the highest school exclusion rates in 
the country. Increasing proportions of adult students, too, needed considerable 
monitoring and support if they were to acquire the necessary study skills. Though 
motivated to learn, some adult students reacted adversely to the thought of formal 
assessment, and this issue had to be dealt with sensitively if they were to gain the 
qualifications towards which they, ostensibly, were studying. At College H there were 
now earlier and more frequent interventions in the event of absence or failure to 
complete coursework, but several staff expressed worries about the difficulty of 
balancing confrontation with sympathy for the personal circumstances of some 
students. 

More generally, effective tutoring was seen to require high levels of judgement and 
inter-personal skills. The involvement of greater numbers of staff in tutorial 
responsibilities therefore had considerable implications for recruitment, training and 
development. There were a number of examples where colleges were giving 
particular attention to these issues. At College J, tutorial policy was supported by a 
whole-college training programme, including at community outreach sites. A 
Personal Tutor Development Group had been established to provide continuing 
support and development, including an up-to-date resource manual. At College C 
there was a short induction programme for all newly appointed tutors, plus a 
voluntary Counselling Skills for Tutors course spread over three halfdays. During 
1999, College F mounted a major cross-college staff development programme for 
tutoring skills in line with the implementation of strategic priorities. 
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Revisions to assessment procedures 

At Coiiege C, there were approximately 13,000 ESOL enrolments per year. Achievement rates had 
been poor, as students did not want to be assessed. Under new arrangements students who were 
afraid of tests were assessed unobtrusively. Assessors examined their work and matched it against 
performance criteria for a college-based OCN qualification. In this way most students now achieved. 



Changes in the tutoriai system 

At Coiiege H, the tutorial system was in the process of changing. In future, all students would have 
an additional one-to-one tutorial once a term in order to look at attendance and punctuality. Tutors 
would track both academic progress and provide pastoral care. Staff development had taken place. 
Lecturers felt the attitude towards tutorials was changing in the college in the light of a recent 
Inspection report. There was now more of an interface between tutors and students and more staff 
were intervening with regard to student absence and late assignment work. 



Target setting and action planning 

Evidence is growing that student motivation can be stimulated and harnessed to 
great effect if challenging but achievable academic targets are set for students 
individually, and progress monitored regularly against them. The use of value added 
measures to predict eventual examination grades based on GCSE point scores has 
had notable impact in raising the achievement on many A-level courses. It also 
appears that such approaches could be transferable to GNVQ qualifications, even 
though the correlations between input and output measures are less pronounced 
than with A-levels (Martinez 2001). 

Unsurprisingly, the two sixth form colleges within the Stage 2 sample (Colleges A 
and B) were making use of ALIS (A-Level Information System) to set A-level grade 
targets - in the former case typically at a grade higher than that predicted! Students 
at both colleges spoke positively about the way such targets made for a constructive 
relationship with their tutors and helped in sustaining their motivation. Value added 
approaches were also being piloted at Colleges C, I and J. At College E students on 
some programmes were involved in major termly progress reviews with their tutors, 
incorporating student self-assessments agreed with, and validated by, their tutor. 
Follow-up action plans were then required. 



Individual target-setting using ALIS 

At College B, students spoke very positively about the relationship established with their tutors who 
helped them to set achievement targets, using ALIS. The teachers help you to realise your potential: 
help is always there if you're willing to take it.' Most courses were modular so external assessment 
enabled regular feedback to be given and students ‘had real targets to aim for'. 



Portfolio development 

At College E, Motor Vehicle students finished each day with a tutorial session that was specifically 
used to develop portfolios and to review the learning that the students had achieved that day. 
Students were wholly supportive of this integrated approach, which seemed to have proved 
successful in demystifying portfolio building for them. 
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Other support 



Though having more immediate relevance to maintaining retention than to improving 
academic performance, Student Services were generally viewed as having a key 
role to play. Several colleges had striven to develop closer links between tutors and 
Student Services with beneficial effect. Tutors were one of the most effective means 
of communicating to students the range and nature of available support (especially 
financial help) so that take-up was optimised. They also played a leading role in 
minimising any perceived stigma attached to the need to receive support. Equally, 
there was evidence that, where personal problems were concerned, students 
preferred not to deal with a member of staff with whom they had a day-to-day 
relationship on academic matters. Hence the need for effective liaison between the 
two functions. 

Specific aspects of Student Services were seen as particularly effective in sustaining 
retention and achievement while widening participation. At College I, for instance, 
the provision of 140 daily childcare places was seen as a major step fonward. The 
well-developed Student Services function at College F supported students in a 
number of ways. It administered the College Access Fund, which supported students 
directly through grants, and indirectly through the purchasing of equipment - for 
example, hairdressing kit. Students were generally expected also to make a notional 
contribution themselves. Student counselling was provided by an external agency 
and support was offered on a fixed term, sessional basis so as to discourage 
dependency. 



K0y chamcteiisllcs - tatoriBf and odter support 

: regular iwiew of students’ academic progress and personal circum^ndes, 
individually as well as in groups 

• the negotiation with individual students of targets and action plans to achieve 
them, making use of value added systems wherever possible 

• close links between tutors and teaching staff 

« pro-active referral to flexible systems of additional learner support, wherever 
necessary 

% ■ close liaison between tutors and Student Services, and other sources of support ; 
on non-academic matters 

• rigorous systems for the selection, training arid appraisal of staff with tutorial 
responsibilities. 
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9 TEACHING AND PEDAGOGY 



Course review mechanisms 

College Quality Assurance (QA) systems and procedures for self-assessment 
generally encompassed regular course review. The main emphasis tended to be on 
the requirement for remedial action plans from course leaders and course teams 
where rates of retention and achievement fell below a pre-determined threshold, and 
where there were no obvious causes lying outside the influence of the staff 
concerned. Regular monitoring of progress against targets was also undertaken, and 
improved MIS was enabling management teams to identify potential problems and 
intervene to tackle them at an early stage. Student feedback was also typically taken 
into account within these processes. 

Less apparent was the rigorous application of such procedures across all courses. 
Teaching staff interviewees in a number of the colleges felt systems were applied 
inconsistently, and hard evidence of the direct impact of QA arrangements on 
teaching and learning practice was not always easy to find. 

There were some examples of reforms in course review procedures and other 
innovative approaches that appeared to be having a positive impact. At College C, 
for instance, all quality systems were reviewed and redesigned following the 
appointment of a new Head Of Quality. Termly achievement reviews were 
introduced, with performance checked against national benchmarks using three-year 
trends. Areas for action were then identified, and the Head of Quality was charged 
with monitoring their implementation. Annual self-assessment reviews were also 
undertaken. Teaching methods were being reviewed, and all students undertook a 
diagnosis of their learning styles to inform this process. College J had introduced a 
Meeting Our Targets (MOT) system that emphasised continual improvement. Among 
its features it involved the analysis of the 50 courses with the best retention and 
achievement rates so that successful strategies could be identified and spread to 
other courses. At College A, major changes to teaching approaches in Mathematics 
in the light of poor rates of retention and achievement had been introduced by a 
newly appointed Head of Department. At College I, grades resulting from lesson 
observations were now made available to course teams (in an anonymised form), 
with a view to encouraging more focused remedial action plans. 



Course quality record holders 

At College E, each course had a designated course quality record holder responsible for the 
completion of a range of course documentation and data in the quality portfolio. This included a 
course review and evaluation together with the development of a course action plan. In turn, this 
informed the self-assessment process, which was located at curriculum department level. Quality 
coordinators had recently been introduced at curriculum department level in order to monitor and 
support completion of course quality records. The college’s SMT had noted the need for more 
detailed monitoring of course quality and action plans. 



O 
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Classroom observation and staff development 

A number of the colleges had placed a major emphasis on classroom observation, 
linked to appraisal and individually tailored programmes of staff development. The 
staff development officer at College C regarded the transformation of teaching 
quality from ‘adequate’ to ‘good’ as the single most effective way of improving the 
performance of students. In order to work, though, these arrangements required the 
confidence of the large majority of staff to whom they were applied. This was unlikely 
to be achieved unless all those who were employed to do the observing were highly 
trained for the task, and seen to be authoritative by those they observed. Ensuring 
that this was the case consumed considerable time and resources. At some of the 
colleges, management teams considered that the defensiveness of staff about the 
process (perhaps understandable) had diluted its effectiveness in identifying and 
tackling remedial needs. At others there was no explicit reference to student 
retention and achievement within the criteria for staff appraisal. 

On the whole, the schemes in place seemed to be much more assiduous at 
identifying and dealing with problem areas than they were at learning from ‘best 
practice’. Many of the teaching staff who were interviewed were not aware of any 
formal systems of sharing practice within their own colleges. 

In general, the features of the best-developed schemes appeared to be: 

• a rigorous lesson observation system applied regularly and consistently across 
all teaching staff 

• the utilisation of teaching observation reports 

• a culture of continual improvement, rather than apportionment of blame, with an 
emphasis on individually-tailored development programmes for a// staff involved, 
irrespective of their level of competence 

• close monitoring of the implementation of development programmes, with 
support for staff whose teaching performance was judged as inadequate 

• identification and dissemination of effective practice, in addition to that in need of 
remedial action. 



Increasing rigour of observation 

At College C, the observation system had been tightened, with automatic follow-up on staff receiving 
an observation grade of 4 or 5, or even a low 3. Remedial work was expected from the staff pending 
re-inspection, and the whole process was now linked with appraisal. 
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Observation and continuous professional development 

At College D, every member of staff was observed. They were given feedback and there was an 
appeals procedure in the event of disagreement. A copy of the report was sent to the personnel 
manager. The previous appraisal scheme had been replaced with a continuous professional 
development (CPD) interview, which identified development needs. 

At College J, identified good practice had been fed into the training provided on lesson observations. 
A paper with a number of recommendations relating to the observation of teaching had recently gone 
to Academic Board. This made reference to national quality standards and the need for professional 
development. Managers at all levels taught. The Professional Development Plan linked to policies and 
strategies designed to raise achievement - for example, the continued provision of in-house 
workshops on improving retention and achievement for 16-19 year olds, ongoing training in lesson 
and tutorial observation, and increasing the number of staff attaining City and Guilds 730 and the 
Certificate in Education. The Plan also aimed to encourage every teacher to develop teaching 
practice against FENTO standards, to provide seminars on excellence in teaching and learning, and 
to fund degrees and masters programmes for specialist staff. 



Teaching and learning styles 

At College I ‘achieving quality through continual improvement’ was cited as a principle underpinning 
the achievement of the college mission. This was mainly effected through staff development. The staff 
development plan continued to include training in a range of teaching and learning styles and 
strategies and to improve areas of weakness in teaching that have been ‘identified through self- 
assessment’. A small pilot training scheme in differential learning and sharing good practice, which 
was carried out in 1998, was being extended to all staff. Workshops would be run at all sites. Good 
practice checklists and a database had been developed in order to promote continual improvement in 
teaching. 



• active involvement of teaching si&fS in course ; 

* teachers accept appropriate accwtaPiiity for student achievement 

• a hgorous, regular and consistent system of lesson observation 

« Mtlllsation of ot^rvatbn reports within agreed systems of staff appraisal 

♦ a culture of continual improvement, supported by individually tailored 
programmes of continuous professional development 

♦ identification and dissemination of effective practice. 
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10 MONITORING, EVALUATION AND FOLLOW-UP 

Information 

There was general agreement that the timeliness, accuracy and usefulness of 
retention and achievement data provided from colleges’ central MIS had improved 
overtime. This had resulted both in raising awareness of staff, and in enabling better 
targeted intervention to take place at an early stage. In most cases, though, there 
was still some way to go, and there were many examples of additional records being 
maintained at departmental level because central systems were still seen as 
incapable of producing data with sufficient speed or accuracy, or in the most helpful 
format. 

The systems that were best regarded by management and staff seemed to be those 
that used college intranet systems to provide data online in a user-friendly format, 
allowing access and interrogation by individual members of course teams. Records 
could be updated daily, so that a high level of currency was maintained provided 
there were no delays in data input. Monitoring could therefore be undertaken on the 
basis of the latest information, enabling potential problems to be identified and acted 
upon before they grew too serious. 

Even with the most sophisticated MIS, though, any positive impact upon student 
achievement depended upon action being taken on the basis of the information 
provided. The point was frequently made by interviewees that problems were known 
of at course team level before they were spotted in MIS reports, but were not always 
tackled as early or as thoroughly as would be desirable because of other pressures. 



User-friendly MIS 

At College C the data obtainable from MIS is now of high quality - the college's FEFC Inspector 
regarded it as good as he had ever seen. The format was user-friendly and was updated every 24 
hours. It was the role of HoDs to identify problem areas. Prior to the installation of the improved 
system, data on retention and achievement had not been presented in a user-friendly format, and 
staff therefore made little use of it. This had now been changed and the intranet MIS format was used 
effectively. Responding to weaknesses was now a priority. 



Accuracy and timeliness 

At College E problems remained in the provision of accurate and timely MIS data against which to 
monitor and plan. There were many instances of discrepancies between course level data and MIS 
data. Some managers had found ways of resolving this difficulty using locally generated data. Others 
however, were not able to challenge the inaccuracy of MIS data with their own figures. Major efforts 
were now being made to ensure the availability of accurate data, so as to give greater impetus to 
monitoring retention and achievement and the use of internal and external benchmarking. 

At College J accurate MIS was recognised as vital to the whole process of monitoring and improving 
retention and achievement. High priority had been given to refining systems and making data 
regularly available to course teams. This priority extended to the examinations section, the staff there 
seeing themselves as a professional team supporting quality assurance. 
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Follow-up of absence and coursework problems 

There was widespread agreement that punctuality and attendance - especially in the 
case of younger students - were accorded much more importance than had usually 
been the case in the past. There was also a consensus that the adoption of ‘tough 
love’ policies in these areas had led to improvements in staying-on rates, sometimes 
markedly so. Immediate follow-up of unexplained absence of 16-19 year olds was 
commonplace, and parents were frequently involved where problems had occurred. 
While the ‘adult atmosphere’ of colleges was an attractive feature for many young 
students, it had come to be accepted that many of them benefited from a significant 
level of order and direction, particularly early on in their studies. For this reason, 
also, there were a number of cases where timetabling had been tightened, with 
directed self-study replacing the concept of ‘free periods’. 

Understandably, perhaps, worries persisted among some interviewees about the 
difficulty of combining even-handedness with sensitivity to individual circumstances, 
and also over the stage at which it became appropriate to have recourse to 
disciplinary procedures. It was noticeable, though, that retention and achievement 
rates were often at their highest within any one college where the routines of a real 
workplace were reproduced, and a strong professional work ethic had been 
established. 

Likewise in the case of problems with academic workload and assessments, there 
was widespread evidence of careful monitoring of individual progress, linked to the 
provision of additional learning support. Here, examples were also seen of the use of 
MIS to identify problems that seemed to affect an unusual number of students on the 
same course, perhaps because of avoidable bottlenecks in the scheduling of 
coursework. There was general agreement that, once a backlog of coursework had 
built up, the students affected were much more likely to withdraw from their studies 
before completion. 



Attendance chasers and homework club 

At College C there was a system of ‘Attendance Chasers’: students were paid to do this and tutors 
did it routinely. Students were encouraged to provide mobile as well as home contact phone numbers. 
On three evenings a week, starting at 4.30 pm, there was a staffed homework club, common to the 
entire curriculum area, so mixing students at Levels 1, 2 and 3. After successful piloting in one 
department, it had been implemented across college. Almost 10% of students made use of it on any 
given day. Most made use of it at some time or other. It was very popular with the students, who 
helped each other. Staff requested students to attend where this was considered necessary, but 
students could self-refer. The club was used least in the first few weeks, but soon caught on. 



Attendance and coursework monitoring 

At College G, student attendance was monitored and was discussed at tutorial reviews. All parents of 
16—19 year olds were contacted if students did not arrive in college by 1 1.30 am. Parents were also 
contacted if students were not producing assignments. In the IT section, 16—19 year olds were rung 
by phone as soon as a session was missed and a signing in system was operated. As a 
consequence, 16—19 attendance had risen from 60% to 80% since the previous year. 




42 



46 



Learning advisers and retention officers 

At College I, lead learning advisers had been established to work with tutors to improve course 
placements and monitor ‘at risk’ students. Site-based retention officers had also been appointed to 
follow up students and liaise with course teams. Periodic retention reports were produced, which 
course teams were required to analyse. Heads of faculty then reported to SMT and action was taken if 
appropriate. Curriculum quality procedures had also been revised. These included a course review 
checklist, the outcomes of which fed into the self-assessment report. The revisions were being 
supported by a whole college staff development day. 



IC$y characten$tfO$ - monitortng, ev^fuation and iniiaw-up 

• MIS capable of providing accurate and up-to-date information on retention and 
achievement, down to course level and by individual student, and In a user- 
friendly fonrnat 

» readily available online access to MIS on the part of managers and teaching sta^ 

• alt staff trained in accessing. Interrogating and making use of MIS data 

• clearly stated policies on attendance and punctuality, tMffi imhiediate follow-up of 
une)^ained absence 

• culture of student obligations as welt as entitlements 

• early idenrif featlon of problems caused by outside pressures, and offer ef support 

• direction and programming of self-study time, especially In the base of youhger 
students 

• involvement cf parents cr other family members In sustaining student support. 



1 1 CONCLUSIONS AND IMPLICATIONS 
Demographic influences on achievement 

Demographic differences in student profiles clearly have an impact upon the relative 
rates of student achievement recorded by different colleges. Generally speaking, 
there is a negative correlation between measures of deprivation and overall rates of 
achievement. Having said this, in itself the demographic make-up of student bodies 
does not appear to be capable of explaining the major part of inter-institutional 
variations in achievement. Colleges with the lowest rates of student achievement are 
below average primarily because their rates are lower for a// their students, not 
because they have higher than average proportions in categories associated with 
below average achievement - even though the latter is usually also the case. 

Nor is there any evidence that we have been able to identify that the remaining 
differences in college achievement rates are connected to any marked degree to 
variations in students’ prior attainments, or in the numbers of qualifications for which 
they are entered. 

The profiles of student bodies, as recorded in the ISR, do not tell the full 
demographic story, as we have acknowledged. Deprivation affects individual 
students in ways that the ISR cannot measure, even indirectly. It has also been 
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postulated that demands on college capabilities grow disproportionately with the 
volumes of student deprivation with which they have to deal, with consequential 
effects on achievement. However, initial analysis of student bodies by postcode - the 
best currently available single indicator of overall deprivation - reveals no evidence 
that deprivation has a significant impact on achievement over and above that 
revealed by ISR comparisons. The relationship between achievement rates and the 
percentages of students from postcodes that trigger the WP uplift is weak, as 
revealed by a number of high achieving colleges that have student profiles 
comprised overwhelmingly of students who trigger the uplift. The line of best fit 
suggests that for every additional 10% of total student numbers that trigger the uplift, 
overall achievement rates fall by around 3% (1997-98 data) or less (1998-99). Even 
if the correlation were strong, though, on this basis it would not account for the bulk 
of the inter-institutional achievement gap. 

A further possibility is that the ethnic mix of student bodies can influence 
achievement rates in ways additional to those accounted for by variations in the ISR. 
It is certainly the case that, on the whole, colleges with the higher levels of 
achievement are those with the smaller proportions of students drawn from minority 
ethnic groups. But as we have pointed out, such students also have relatively higher 
levels of achievement at the colleges in question. As with deprivation, at present we 
have no hard statistical evidence that demands on colleges grow disproportionately 
with the numbers of students from minority ethnic groups, with consequential effects 
on overall achievement rates. This is not to deny any deleterious effects of racism, 
negative stereotyping and the like, but simply to query why their impact on the 
achievement of individual students should vary in this way. Racism would appear to 
be no less common in parts of the country where the minority ethnic presence within 
student bodies is very small, and where there are fewer peer group support 
mechanisms to help its victims to cope. 

Having said this, there may be inter-institutional differences in the socio-economic 
profiles of the same minority ethnic groups or in the extent to which English is a first 
language, or both, which are not readily apparent from the ISR. These might help to 
explain some of the remaining achievement gap. 

In summary, therefore, demographic variations appear to account at most for no 
more than half of the inter-institutional differences in student achievement, according 
to the measures currently available. This suggests that all high WP colleges with 
overall achievement rates below the average for that group should be capable of 
improving them to the current median level, at the very least. Further investigations 
of the influences of deprivation and ethnicity are needed, though, before we can fully 
exclude the possibility that they could also explain a significant element of the 
remaining achievement gap. 
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Institutional performance 

On the basis of our examination of the demographic evidence, therefore, our 
supposition is that a substantial part of the achievement gap must stem from factors 
that lie within the direct influence of colleges - that is, as a result of differences in 
institutional ethos, systems, procedures and practices that affect student 
performance. Evidence already assembled concerning influences on student 
retention, together with the reports from the FEFC Inspectorate, and from the 
National Audit Office, supports this hypothesis. Colleges with low achievement rates 
are by no means confined to those serving areas of high deprivation - several with 
average or low deprivation student profiles display achievement rates that are well 
within the bottom quartile. Furthermore, year-on-year improvements have generally 
been most pronounced among the colleges with the lower achievement rates - a 
pattern that is consistent with the hypothesis that a significant element of student 
achievement is directly susceptible to college-level interventions. 

Our study of 10 colleges that formed Stage 2 of this project suggests that, in general 
and taking account of the variations in the profiles of the student bodies, those with 
the better performance in terms of overall achievement rates during the period 1 GOO- 
GO had for whatever reason made the improvement of retention and achievement a 
major strategic priority at an earlier stage and had been more successful at 
embedding a culture of continual improvement at all levels. In these cases there was 
evidence extending over a longer period of: 

• the high priority accorded to raising student achievement within the college 
strategic plan 

• regular consideration of the issue by the SMT and governing body 

• the establishment of achievement targets, against which progress was 
monitored, and the use of benchmarks 

• effective communication to staff of the central importance of raising student 
achievement 

• a culture of high expectations of both staff and students 

• the allocation of responsibility and authority at curriculum leader and course team 
level 

• regular monitoring and action planning via face-to-face meetings between 
managers and staff 

• encouragement of bottom-up initiatives within a college-wide strategy framework 

• review of teacher performance, supported by tailored staff development. 

Otherwise, though, there was no clearly obvious pattern that linked differences in 
college ethos and practice to variations in their achievement rates. In coming to this 
conclusion, we note: 

• the difficulty of making due allowance for any disproportionate impacts of merger 
and restructuring that affected some of the colleges concerned 

• the variations in achievement rates within as well as between colleges 

• the more stringent highlighting of problem areas for retention and achievement in 
the second round of FEFC Inspections 

• most importantly, the changes that a number of the Stage 2 colleges had put in 
place since the last (1GG8-GG) ‘official’ set of achievement figures. 
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We have seen that, nationally, colleges with below average achievement have been 
helping to ‘close the achievement gap’ by improving their position at a relatively 
faster rate. The same trend appeared to be apparent within the group of 10 colleges 
that we visited. Though we cannot tell for certain that it will be continued when the 
fully audited figures for subsequent years become available, there was every sign of 
a generally high level of commitment to improving student achievement, at least on 
the part of management teams and governing bodies. As we have noted in the 
preceding sections of this report, a number of the most interesting innovations are 
now in place colleges with the greatest cause for concern about their achievement 
rates. 



Raising student achievement 

Despite the absence of clear links between specific interventions at college level, 
and precisely predictable impacts on student achievement, we have found it possible 
to arrive at some measure of consensus concerning effective improvement 
strategies. In combination, the reports of the consultants, based on the opinions of 
the managers, teaching staff and students whom they had interviewed, identified a 
number of key characteristics where confidence levels in their effectiveness 
appeared to be highest, as summarised at the end of each of the preceding sections 
4-10 of this report. 



The way forward 

No one college, of course, can be said to excel in all the key characteristics that we 
have identified. Equally, there is unlikely to be a college that is totally devoid of 
effective practice, and there are signs of the spread of more consistent approaches 
to quality improvement. Some areas remain less developed than others, though: 

• MIS, where despite improvements there is still a wide gap between the best 
practice and the level of development that is typical 

• changes in course delivery in response to the more rigorous identification of 
retention and achievement problem areas.; in particular, the concepts of inclusive 
learning still have some way to go before they are likely to impact fully on the 
majority of students 

• sharing and learning from effective practice: currently, colleges seem much 
better attuned to identifying weaknesses and acting upon them, than in building 
on their own points. 

Our main findings are consistent with others arising from research undertaken by the 
Learning and Skills Development Agency. Readers are recommended to refer to the 
list of references that follows for publications containing this evidence, as well as 
guidance on improvement strategies. 

The ways in which student achievement can be raised most effectively remain a 
complex issue. More work is required to understand fully the aspects in which 
demographic factors act as barriers before we can be certain that their impact is not 
greater than we believe on the basis of the evidence currently available. At present 
though, it appears that the efforts at college level are already bringing about a 



noticeable narrowing in the inter-institutional achievement gap. As we have 
indicated, such efforts are by no means straightforward, and place considerable 
demands upon colleges. Ultimately, though, we believe that it ought to be possible to 
reduce the achievement gap to half its present size, or less. 
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